Joint Multi-Dimension Pruning via Numerical Gradient Update

We present joint multi-dimension pruning (abbreviated as JointPruning), an effective method of a network on three crucial aspects: spatial, depth and channel simultaneously. To tackle these naturally different dimensions, we proposed general framework by defining seeking the best vector (i.e., numerical value layer-wise number, spacial size, depth) construct unique mapping from to pruned structures. Then optimize with gradient update model optimization process. overcome challenge that there is no explicit function between loss vectors, self-adapted stochastic estimation path through vectors enable efficient update. show strategy discovers better status than previous studies focused individual dimensions solely, our optimized collaboratively across in single end-to-end training it more exhaustive methods. Extensive experiments large-scale ImageNet dataset variety architectures MobileNet V1&V2&V3 ResNet demonstrate effectiveness method. For instance, achieve significant margins 2.5% 2.6% improvement over state-of-the-art approach already compact V1&V2 under extremely large compression ratio.